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This paper aims at proving asymptotic stability of the radial stationary solution of a free 
boundary problem modeling the growth of nonnccrotic tumors with fluid-like tissues. In a 
previous paper we considered the case where the nutrient concentration ct satisfies the sta- 
tionary diffusion equation Act = /(ct), and proved that there exists a threshold value 7* > 
for the surface tension coefficient 7, such that the radial stationary solution is asymptotically 
stable in case 7 > 7* , while unstable in case 7 < 7* . In this paper we extend this result to the 
case where ct satisfies the non-stationary diffusion equation edta = Act — /(ct). We prove that 
for the same threshold value 7* as above, for every 7 > 7* there is a corresponding constant 
£0(7) > such that for any < e < £0(7) the radial stationary solution is asymptotically 
stable with respect to small enough non-radial perturbations, while for < 7 < 7* and e 
sufficiently small it is unstable under non-radial perturbations. 

AMS subject classification: 35R35, 35B35, 76D27. 
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1 Introduction 

This paper is concerned with the following free boundary problem modelling the growth of 
tumors with fluid-like tissues: 



Abstract 




t > 0, 




V • v = g(a) in Q(t), t > 0, 

uAv + Vp - -V(V • v) = in t>0 
3 

a = a on dfl(t), t > 0, 
T(v,p)n = -7Kn on ffi(t), t > 0, 
V n = v • n on dtt(t), t > 0, 









(1.9) 
(1.10) 



(1.7) 



(1.8) 



where a = a{t,x), v = v(i, x) (= x), V2(t, x), Vs(t, x))) and p = p(t,x) are unknown func- 
tions representing the concentration of nutrient, the velocity of fluid and the internal pressure, 
respectively, / and g are given functions representing the nutrient consumption rate and tumor 
cell proliferation rate, respectively, and Cl(t) is an a priori unknown bounded domain in M 3 
representing the region occupied by the tumor at time t. Besides, e, u, a and 7 are positive 
constants, among which e is the ratio between typical, v is the viscosity coefficient of the tumor 
tissue, 7 is the surface tension coefficient of the tumor surface, and a is the concentration of 
nutrient in tumor's host tissues, k, V n and n denote the mean curvature, the normal velocity 
and the unit outward normal, respectively, of the tumor surface d£l(t), and T(v,p) denotes the 
stress tensor, i.e., 



where I denotes the unit tensor. We note that the sign of the mean curvature k is defined such 
that it is nonnegative for convex hyper-surfaces. Without loss of generality, later on we assume 
that v = 1 and a = 1. Note that the general situation can be easily reduced into this special 
situation by rescaling. As in [14J, throughout this paper we assume that / and g are generic 
smooth functions satisfying the following assumptions: 

(Al) / G C°°[0, 00), f'(a) > for a > and /(0) = 0. 

(A2) g G C°°[0, 00), g'(a) > for a > and g(a) = for some a > 0, 

(A3) a < a. 

The above problem is a simplified form of the tumor models proposed by Franks et al in 
literatures [lj-[7], which mimic the early stages of the growth of ductal carcinoma in breast, and 
was first studied by Friedman in [8 J . Local well-posedness of this problem in Holder spaces has 
been established by Friedman [8J in a more general setting. Moreover, in [8] it is also proved 
that, for the special case f(cr) = Act and g(a) = n(cr — a), the problem (1.1)— (1.10) has a unique 
radially symmetric stationary solution (a s ,v s ,p s , Cl s ). In [11] Friedman and Hu proved that there 
exists a threshold value (/u/7)* such that in the case /x/7 < (/u/7)* this radial stationary solution 
is linearly asymptotically stable, i.e. the trivial solution of the linearization at (cr s , v s ,p s , O s ) of 
the original problem is asymptotically stable, and in the case /i/7 > (/Vt)* this stationary 
solution is unstable. However, whether or not in the case fi/j < (/i/7)* this stationary solution 
is asymptotically stable, namely, whether or not (<r s , v s ,p s , O s ) is asymptotically stable under 
arbitrary sufficiently small non-radial perturbations, which is the Problem 3 of [8] (see also the 
Open Problem (i) in Section 2 of [9]), was not answered by these mentioned literatures. 

In a previous work (see [14]) we studied the above problem for the model simplified from 
(l.l)-(l.lO) by taking e = 0, and proved that there exists a threshold value 7* > for the surface 




(1.11) 



tension coefficient 7, such that in the case 7 > 7* the radial stationary solution is asymptotically 
stable with respect to small enough non-radial perturbations, while in case 7 < 7* this stationary 
solution is unstable under non-radial perturbations. The aim of the present work is to extend 
this result to the case where e is non-vanishing but small. We shall prove that for the same 
threshold value 7* as above, for every 7 > 7* there is a corresponding constant £0(7) > such 
that for any < e < £0(7) the radial stationary solution is asymptotically stable with respect 
to small enough non-radial perturbations, while for < 7 < 7* and £ sufficiently small it is 
unstable under non-radial perturbations. To give a precise statement of our main result, let us 
first introduce some notations. 

As in [H], we denote by (a s , v s ,p s , fi s ) the unique radial stationary solution of (1.1)— (1.8), 
i.e., Q s = {3;El 3 : \x\ < R s } and 

a s = a s (r), v s = v s (r)-, p s = p s (r) for x e O s , 

and for any xq € M 3 we denote by (ffy, Vr^i,^^], £l\ xo ]) the stationary solution of (1.1)-(1.8) 
obtained by the coordinate translation x — > x + xq of the stationary solution (a s ,v s ,p s ,Q s ). 
Given p £ C 1 (0O s ) with H/ollc 1 (<9ri s ) sufficiently small, we denote by Q p the domain enclosed by 
the hypersurface r = R s + p(£), where £ E <9f2 s . Since we shall only be concerned with small 
perturbations of the stationary solution (a s , v s ,p 3 , £l s ), there exist functions p{t) (= p(£,t)) and 
Po (= Po(£)) 011 such that U(t) = Q p (t) and S7o = Q po - Using these notations, the initial 
condition (1.10) can be rewritten as follows: 

p(£,0)=p o (£) for £e£>O s . (1.12) 

The solution (c, v,p, £1) of the problem (1.1)— (1.9) will be correspondingly rewritten as (a, v,p, p), 
and the radially symmetric stationary solution (a s , v s ,p s , J7 S ) will be re-denoted as (a s , v s ,p s , 0). 
The main result of this paper is the following theorem: 

Theorem 1.1 Assume that Assumptions (Al)-(A3) hold. For given m € N ; m > 3, and 

< 6 < 1, we have the following assertion: There exists a positive threshold value 7* such that 
for any 7 > 7*, the radially symmetric stationary solution (o~ s ,v s ,p s ,0) is asymptotically stable 
for small e in the following sense: There exists 8q > such that for any < e < £q there exists a 
corresponding constant e > 0, such that for any po S C m+e (d£l s ) satisfying \\po\\c m - 1 + e (dCi 3 ) < e > 
the problem (1.1)-(1.9) has a unique solution (a,v,p,p) for all t > 0, and there are positive 
constants u, K independent of the initial data and a point x$ G M 3 uniquely determined by the 
initial data, such that the following holds for all t > 1: 

lk(')*) ~ a [x }\\c m + e (Q(t)) + ll v (')0 _ v [x ]llc m - 1 + 9 (c(t)) 
+1 \p(-,t) ~ P[x ] I lc™- 2 + 9 (QW) + I |p(-> *) - P[x ] I \c™+»(dn s ) < Kb'"*. (1.13) 

For 7 < 7* and £ sufficiently small, the stationary solution (a s ,v s ,p s ,0) is unstable. □ 

As in [14] we shall use a functional approach to prove this result, namely, we shall first 
reduce the problem (1.1)— (1.10) into a differential equation in a Banach space, and next use the 
geometric theory for differential equations in Banach spaces to study the asymptotic behavior 
of the reduced equation. However, unlike the case e = in which the reduced equation is a 



scalar (first-order) nonlinear parabolic pseudo-differential equation on the compact manifold § 2 
which does not have a boundary, in the present case £ / the reduced equation is a system 
of equations, one of which has a similar feature as the equation in the case e = 0, while the 
other of which is defined on the domain £l s complemented with a Dirichlet boundary condition. 
This determines that in the present case we are forced to deal with a number of new difficulties. 
The first difficulty lies in computation of the spectrum of the linearized operator, because we 
now encounter a matrix operator which is not of the diagonal form. To overcome this difficulty 
we shall use a technique developed in [5j to show that the linearized operator is similar to a 
small perturbation of a matric operator possessing a triangular structure. The second difficulty 
is caused by the Dirichlet boundary condition, which determines that we cannot find a suitable 
continuous interpolation space as our working space to make the center manifold analysis. More 
precisely, as in the case e = 0, is an eigenvalue of the linearized operator, so that the standard 
linearized stability principle does not apply. In the case e = 0, this difficulty is overcome with 
the aid of the center manifold analysis technique developed in [3j. Since this technique requires 
that the working space must be a continuous interpolation space, it fails to apply to the present 
case £ / 0. To overcome this difficulty we shall use the idea of Lie group action developed in [2] 
and apply Theorem 2.1 of [2] to solve this problem. 

The structure of the rest part is as follows. In Section 2 we first convert the problem into an 
equivalent initial-boundary value problem on a fixed domain by using Hanzawa transformation, 
and next we further reduce it into a differential equation in a Banach space. In Section 3 we 
study the linearization of (1.1)— (1.8) at the radial stationary solution, and compute the spectrum 
of the linearized operator. In the last section we give the proof of Theorem 1.2. 

2 Reduction of the problem 

In this section we reduce the problem (1.1)— (1.10) into a differential equation in a Banach space. 
For simplicity of the notation, later on we always assume that R s = 1. Note that this assumption 
is reasonable because the case R s ^ 1 can be easily reduced into this case after a rescaling. It 
follows that 

n s = B 3 = {x G M 3 : |x| < 1} and dn s = § 2 = {x 6 I 3 : |x| = 1}. 

Let m and 6 be as in Theorem 1.1. We introduce an operator II G L(C m+e (§ 2 ), C m+9 (B 3 )) 
as follows: Given p G C m+9 (S 2 ), we define u = U(p) G C m+9 (E 3 ) to be the unique solution of 
the boundary value problem 

Au = in B 3 , u = p on § 2 . (2.1) 

It is clear that IT G L((7 m+6> (§ 2 ), C™ +6, (B 3 ), and IT is a right inverse of the trace operator, i.e., 
we have tr s2 (ll(u))) = u for any u G C m+e (S 2 ). Let E G L(C m+e (B 3 ), BUC m+e (R 3 )) be an 
extension operator, i.e., E has the property that E(u)(x) = u(x) for any u G C m+6, (B 3 ) and 
x G B 3 . Here BUC m+9 (M. 3 ) denotes the space of all C m functions u on R 3 such that u itself and 
all its partial derivatives of order< m are bounded and uniformly #-th order Holder continuous 
in M 3 . We denote ni = £oE Then clearly n x G L(C m+e (S 2 ), BUC m+e (M 3 )), so that there 



exists a constant Co > such that 

l|ni(p)|| B tK7»>+»(R3) < CoMcm+e^ for p G C m+e (S 2 ). (2.2) 

Take a constant < <5 < min{l/6, l/(3Co)} and fix it, where Co is the constant in (2.2). We 
choose a cut-off function \ £ C°°[0, oo) such that 

os * si - faM>w, md k'w'^l' (23) 

We denote 

0™ +e (S 2 ) = {p G C m+e (S 2 ) : ||p|| c ^ (s2 ) < 5}. 
Given p G 0™ +e (§ 2 ), we define the Hanzawa transformation & p : R 3 — > M 3 as follows: 

= x + x(r- l)ni(p)(x)^ forzGK 3 . (2.4) 

Using (2.2) and (2.3) we can easily verify that 

<& p G Diff m+6 '(]R 3 , M 3 ) and <b p (x) = x if dist(x, S 2 ) > 35. 

We define <p p = $p|g 2 and T p = lm(0 p ), and denote by Q p the domain enclosed by T p . Clearly, 

<p p (uj) = [1 + p(u)]u for w G S 2 . 

Thus, in the polar coordinates (r,uj) of M 3 , where r = |x| and to = x/\x\, the hyper-surface T p 
has the following equation: r = 1 + p(u;). 

Next, given p G C([0, T], 0™ +f? (S 2 ), for each t G [0,T] we define T p (t) = T p(t) and Q p (t) = 
£lp(t) • Since our purpose is to study asymptotical stability of the radially symmetric stationary 
solution, later on we always assume the initial domain 0,q lies in a small neighborhood of 0, s . 
More precisely, we assume To := 

Let p be as above, and let & p be the i-th component of & p , i = 1, 2, 3. We denote 

[0*^- = a£(s) = [^(x)]-. 1 (i,j = 1,2,3), 

G„(x) = det(D* p (x)) for x G M 3 , 

= |0 P | 2 ^1 + |V^ P | 2 for w G S 2 , 

where Va; represents the orthogonal projection of the gradient onto the tangent space 
TL^S 2 ) 1 ). Here and hereafter, for a matrix A we use the notation Aij to denote the element of 

1 - ) In the coordinate ui — uj($,ip) = (sini? cos ip, sin ■& sin <p, cos #) (0<i9<7r, 0<iy9< 27r) of the sphere we have 

Vo>/(w) = (cos 1? cos <£, cos ■& sin — sin#)d,j/(a>(#, y>)) H — r— ^ (— siny>, cos Q)d v f{uj{'d, if)). 

df 1 

Note also that V x f = -^-u + -V„/. 

or r 



A in the (i,j)-th position. From (2.3) we see that [p -► <f> p ] G C°°(0™ +f? (S 2 ), DifF m+e ( 
Thus we have 

>-a£] GC-(0™ +f? (S 2 ),C m - 1+e (R 3 )), *,j = l,2,3, 
< [p -» G p ] G C°°(0^ +e (S 2 ), C m - 1+e (R 3 )), (2.5) 
, [p -> flp] G C°°(0™ +e (S 2 ), C m ' 1+e (S 2 )). 
We now introduce four partial differential operator A(p), B(p), B(p)- and B(p)® on R 3 as follows: 

A(p)u(x) = a^(x)dj (a^ k (x)dku(x)) for scalar function u, 

B{p)u{x) = (^a^j(x)dju(x),a2j(x)dju(x),a^j(x)dju(x)) for scalar function u, 

B(p) ■ v(x) = a?j(x)djVi(x) for vector function v = (vi,V2,vs). 

B(p) ® v(x) = (a p ik {x)dkVj(x)) for vector function v = (t>i, t>2, ^3). 

Here and hereafter we use the convention that repeated indices represent summations with 
respect to these indices, and dj = d/dxj, j = 1,2,3. These definitions can be respectively 
briefly rewritten as follows: 

A( P )u = (A(u o (&; 1 )) o d> p , B(p)u = (V(« o cD; 1 )) o d> p , 

£(p) • v = (V • (v o o ^(p) ^ v = (V (v o Sj 1 )) o $ p . 

By (2.5) we have 

'[p -» .A(p)] G C oo (O™ +0 (S 2 ),L(C ,m + f? (B 3 ),C m - 2+e (B 3 ))), 
[p -» S(p)] G C 00 (0™ +9 (S 2 ),L(C m+9 (B 3 ), (C m - 1+9 (B 3 )) 3 )), 
[p-><B(p)-] G C 0O (0™ +e (S 2 ),L((C m+e (B 3 )) 3 ,C m - 1 + e (B 3 ))), 
[ [p -> B(p)®] G C°°(0^ +e (S 2 ), L((C m+e (B 3 )) 3 , (C m - 1+e (B 3 )) 3x3 )). 



(2.6) 



Next we introduce the boundary operator T>(p): (C r 



))3 ^ C m-l+e^ 



P(p)v = tr§ 2 (v) • [w 



and the bilinear operator C(p): 



1+P 

x (C m - 1+e (B 3 )) 3 



Vu,p], 



Qm—l+d/ 



C(p)[u,v] = X (r ~ l)U 1 (V(p)y)B(p)u • e r . 

Here and hereafter we use the notation e r to denote the vector function on R 3 \{0} defined by 
e r (a;) = u(x) = x/r. Note that since x( r — 1) = for < r < 1 — 3(5, we see that C(p)[u, v] is a 
well-defined function on B . Again, by (2.5) we have 



[p — > P] g C 00 (Of +e (S 2 ),L((C ,m - 1+e (B 3 )) 3 ,C m - 1+e (S 2 ))), 
>->C(p)] G C°°(0^ +e (S 2 ),£L(C m + e (B 3 ) x (C m - 1+e (B 3 )) 3 ,C m - 1+e (B 3 ))). 



(2.7) 



Here the notation BL{- x •,•) denotes the Banach space of bounded bilinear mappings with 
respect to the indicated Banach spaces. 



Let n and k be respectively the unit outward normal and the mean curvature of T p (see 
(1.5)). We denote 

Hp(x) = n(4>p(x)), K p (x) = n(4> p (x)) for x € § 2 . 

A direct computation shows that 

~ = x-[(D^ p (x))- 1 ] T = a^xjej 
P \x- [{D$p{x))- 1 ] T \ H^xjeil' 

where 

ei = (1,0,0), ea = (0,1,0), e 3 = (0,0,1), 
and j 

By (2.5) we have 

\p -» k p ] e c°°(o™ +e (s 2 ),c m - 2 + e (s 2 )), 

[p -» n p ] G C°°(0™ +e (§ 2 ), (C m ^ 1+e (§ 2 )) 3 ). (2 ' 8) 
As in [H] we introduce the following vector functions: 

wi(z) = (0, x 3 , -x 2 ), w 2 (x) = (~X 3 , 0, xi), w 3 (x) = (x 2 , -xi, 0). 

Then clearly v x x = (v • wi,v • w 2 , v • W3). 

Let T be a given positive number and consider a function p : [0, T] — > 0™ +e (§ 2 ). We 
assume that p € C([0, T], 0™ +6, (§ 2 )). Given a such p, we denote 

r p (t) = r p(t) , n p (t) = n pW (o<t<r). 

Finally, for a, v and p as in (1.1)-(1.9), we denote 

a = a o <3? p , v = vo$ p , p = poQ p . 

We also denote = Wj o <3? p , j = 1, 2, 3. 

Using these notations, we claim that the Hanzawa transformation transforms the equations 



(1.1)-(1.10) into the following equations, respectively: 

ed t a - A{p)a - eC{p) [a, v] = -/(of) in B 3 xR + , (2.9) 

B(p) • v = g(a) in B 3 x R + , (2.10) 

-A{p)\ + B(p)p- -JB(p)(B(p) • v) = in B 3 x R + , (2.11) 

of = 1 on S 2 x R+, (2.12) 

T p (v,p)H p = -7K p H p on S 2 x M + , (2.13) 

d t p = V(p)v on S 2 x R+, (2.14) 



/ v(x)G p (x)dx = 0, t > 0, (2.15) 

J\x\<l 

/ v(x) • w p Ax)G p (x)dx = 0, j = 1,2,3, t > 0. (2.16) 

of(i, 0) = 3F (a?) for x G B 3 , (2.17) 
p(w,0)=poM forwGS 2 . (2.18) 

Here T p (v,p) = [B(p) ® v + (£(p) ® v) T ] - [p + (2/3)B(p) • v]I. 

Indeed, it is immediate to see that under the Hanzawa transformation, the equations (1.2)— 
(1.5) and (1.7)-(1.9) are respectively transformed into the equations (2.10)-(2.13) and (2.15)- 
(2.17), and (2.18) is a rewritten form of (1.10). In what follows we prove that (1.1) and (1.6) 
are transformations of (2.9) and (2.14). 

Let ip p (x,t) = r — 1 — p(u,t), where r = \x\ and u = x/\x\. Then x G T p (t) if and only if 
tp p (x,t) = 0. It follows that the normal velocity of T p (t) is as follows (see [3]): 

v n (x,t)= ® tP ^ ( \ forxer p (i), t>0. 

\\7 x ipp{x,t)\ 

Moreover, n(x,t) = V x tlp p (x,t)/\'S/ X ipp(x,t)\. Hence (1.6) can be rewritten as follows: 
d t p(u),t) = v(x,t) ■ V x ifj p (x,t) for x G T p (t), t > 0, 



dip 1 

where u> = x/\x\. Since V x ip p = ~rr~ UJ H — V^VV' we see that after the Hanzawa transformation, 

or r 

this equation has the following form: 



d t p(u,t) = \(u,t) 



■ _ V u p{uj,t) 
" l+p(w,*) 



for uj G S 2 , t > 0. 



Recalling the definition of the operator T>(p), we see that the equation (2.14) follows. Next, by 
differentiating the relation a = a o <3? p in i and using the equations (1.1), (2.4) and (2.14) we see 
that 

d t a=d t a o $ p + 3^ • (V x ct o $ p ) 

=^Afj o $ p - _/(<r o p ) + x ( r - l)ni(^p)e r • (y x o o $ p ) 
=^(p)5 - -/(?) + X (r - l)n!(P(p)v)e r • B{p)a 

J-A{p)a--J{a)+C{p)[a,v)]- 

Hence (2.9) follows. 

The above deduction yields the following lemma: 

Lemma 2.1 If (a, v,p, p) is a solution of the problem (1.1)— (1.10), then by letting a = 
croQp, v = vo<&p andp = po& p , we have that (a, v,p, p) is a solution of the problem (2.9)-(2.18). 
Conversely, If (a,v,p,p) is a solution of the problem (2.9) -(2. 18), then by letting a = a o Qp 1 , 
v = v o <1>~ 1 andp = po§p~ 1 , we have that (a, v,p, p) is a solution of the problem (1.1) -(1. 10). 



Proof: The above deduction shows that if (cr, v,p, p) is a solution of (l.l)-(l.lO), then 
(a,v,p,p) satisfies (2.9)-(2.18). The converse can be similarly verified. □ 

We now proceed to reduce the problem (2.9)-(2.18) into evolution equations only in a and 
p. The idea is to solve equations (2.10), (2.11), (2.13), (2.15) and (2.16) to get v and p as 
functionals of a and p, and next substitute v obtained in this way into equations (2.9) and 
(2.14). Thus, for given p G 0™ +e (S 2 ) we consider the following boundary value problem: 



B(p) ■ v = ip in B 3 , 

-A(p)v + B(p)p = g in B 3 , 
T p (v,p)n p = h on § 2 , 

/ v(x)Gp(x)dx = 0, 

J\x\<l 

[ v{x)-w P {x)G p {x)dx = 0, j = 1,2,3, 

J\x\<l 



(2.19) 

(2.20) 
(2.21) 

(2.22) 
(2.23) 



where ip G C^^B 3 ), g G (C m ^ 2 ^(B 3 )) 3 and h G (C m ^ 1 ^(S 2 )) 3 for some 0< k <m-2. 

Lemma 2.2 Let 5 be sufficiently small and let p G 0™ +e (S 2 ) be given. A necessary and 
sufficient condition for (2. 19) -(2.23) to have a solution is that ip, g and h satisfy the following 
relations: 



1 



X <1 



g(x)--B(p)p(x))-w P (x)G p (x)dx+ / h(x)-w p (x)H p (x)dS x = 0, j = 1,2,3, (2.24) 



X =1 



/ (g(x) - \B(pMx)) ■ e 3 G p {x)dx + / h(x) • e 3 H p {x)dS x = 0, j = 1, 2, 3. (2.25) 

J\x\<l <J J|x|=l 

If this condition is satisfied, then (2.19)-(2.23) has a unique solution (v,p) G (C m_fe+e '(B 3 )) 3 x 



C 



m— fc— 1+6 



Moreover, we have v = V{p)p + Q(p)g + R(;o)h ; where 



m-2 



P6 f] C' 0O (0™ +e (S 2 ),L(C7 m - fc ~ 1+e (B 3 ),(C' m - fc+9 (B 3 )) 3 ) 



fc=0 
m-2 



Q G Pi C°°(0™ +e (S 2 ),L((C r ' 



(B°)) 3 ,(C m - fc+e (B 3 )) 3 )), 



A;=0 
m-2 



Re f] C 00 (0™ +e (S 2 ),L((C m - /c - 1+e (S 2 )) 3 ,(C m - fc+e (B 3 )) 3 )). 



fc=0 



(2.26) 



Proof. See Lemmas 2.3 and 2.4 of [14j . □ 

Now, let a G C m+e (B 3 ), p G Of +e (S 2 ) and we consider the system of equations (2.10), 
(2.11), (2.13), (2.15) and (2.16). These equations can be rewritten in the form of (2.19)-(2.23), 
with 

<p = g(a), g=-B(p)g(a), h = - 7 7i p n p . (2.27) 



As was shown in |14j . the relations (2.24) and (2.25) are satisfied by these functions. Besides, 
it is obvious that ip G C m+e (M 3 ) C C m " 2 + e (B 3 ) and g G (C m - 1+0 (B 3 )) 3 C (<7 m - 3+e (B 3 )) 3 . 
Furthermore, by (2.8) we see that h G (C m ~ 2+e (§ 2 )) 3 . Hence, by Lemma 2.2 (with k = 1) we 
infer that these equations have a unique solution (v,p) G (C m_1+e (B 3 )) 3 x C m_2+e (B 3 ), and 



v = V(a, p) = V( P )g{a) + -Q(p)B(p)g(a) - 7 B.{p)(]C(p)Af{p)). 
where K,{p) = k p and M(p) = n p . We note that 

tc g c°°(o™ +9 {s 2 ), c m - 2+e (§ 2 )), g c°°(o™ +9 (s 2 ), (c m - 1+9 (s 2 )) 3 ). 



(2.28) 



(2.29) 



Substituting the above expression of v into (2.9) and (2.14), and introducing operators T : 
C m + e (M 3 )xO^ +e (S 2 ) -» C m " 2+e (B 3 ) and£ : C m+9 (B 3 ) xO™ +9 (§ 2 ) -» C7 m - 1+9 (§ 2 ) respectively 
by 

^(5, p) = - e A{p)a - X (\x\ - 1)11! (P(p)V(a, p))£(p)a • e r - -J{a), 



G(a, p) = V(p)V(a, p) = tr §2 [V(a, p)] 



1 



1 + P 



(2.30) 
(2.31) 



and p G Oj b+ (§ 2 )) ; where as before u represents the variable in § 2 and 



(for a G C m+e { 

u>(x) = x/\x\ for x G B 3 \{0}, we see that the problem (2.9)-(2.18) is reduced into the following 
problem: 

'd t a = F{a,p) in B 3 x R+, 



dtp = Q(cr, p) on S 2 x 
a = 1 on S 2 x K + , 
a\t=o = (T in B 3 , 
^p\t=o = po on S 2 . 



(2.32) 



We summarize: 



Lemma 2.3 Let (<7,v,p>,p) be a solution of the problem (2.9)-(2.18). Then (a,p) is a 
solution of the problem (2.32). Conversely, if (a,p) is a solution of (2.32), then by letting (y,p) 
be the unique solution of the problem (2. 19) -(2.23) in which ip, g and h are given by (2.27), we 
have that (a,v,p,p) is a solution of (2.9)-(2.18). □ 

The problem (2.30) can be rewritten as an initial value problem of a differential equation 
in a Banach space. For this purpose we denote 



X = C m - 2+e (B 3 ) x C m - 1+9 (S 2 ), 



[ = (C m+e (B d ) nC (B )) x C m+e (S 2 ), 



(C m+e (B 3 ) n Co(B J )) x Of +ti (8 z ), 



where C (B 3 ) = {u G C(B 3 ) : n|g 2 



domain ©a (i.e., F : 



0}, and define a bounded nonlinear operator F in X with 
X) as follows: 



¥(U) = {F{u + 1, p),G(u + 1, p)) for U = (u, p) G O s . 



(2.33) 



Then (2.32) can be rewritten as an initial value problem of a differential equation in X: 

'§= ¥ ^ fOT * >0 < (2.34) 
U\ t =o = Uq, 



where Uq = (<Jq — 1, po). The relation between solutions of (2.32) and (2.34) is that U = (a — 1, p). 
We note that Xq is not dense in X. 



3 Linearization of ¥{U) 

Let (a s ,v s ,p s , R s ) be the radially symmetric stationary solution of the problem (1.1)— (1.10) 
(recall that R s = 1) and denote U s = (a s — 1,0). Then U s is a stationary solution of the 
differential equation in (2.31), so that ¥(U S ) = 0. 

From (2.6), (2.7) and (2.29) it can be easily seen that F € C°°(0,X), where O is regarded 
as an open subset of Xo- It follows that the Frechet derivative D¥ E C°°(0, L(Kq, X)). In this 
section we first derive a useful expression of D¥(U S ), and next use it to prove that D¥(U S ) is 
an infinitesimal generator of an analytic semigroup in X with domain Xq. 

By (2.33) we see that for V = (v,r]) € Xo, we have 

D¥(U S )V = (D^(a s , Q)v + D p F(a s , 0) V , D^g(a s ,0)v + D p G(a s , Q) v ), (3.1) 

where D^T and D p T represent Frechet derivatives of T(o,p) in a and p, respectively, and 
similarly for D^Q and D p Q. In what follows we deduce expressions of these Frechet derivatives. 

We first note that, clearly, 

A(0)u = Au, B(0)u = Vu, B{0) -v = V ■ v, B(0)®v = V®v, (3.2) 

P(0)v = tr s2 (v)-n , /C(0) = H p \ p=0 = 1, tf(0) = n p \ p=0 = n , (3.3) 

G p (x)\ p=0 = l, H p (x)\ p=0 = l, w p j (x)\ p=0 = w j (x). (3.4) 

In (3.3) no denotes the unit outward normal of the unit sphere S 2 , and this notation will be 
used throughout the remaining part of this paper. We also denote 

C — 1 w e,? — w; 
M(f7) = x(r-l)IIi(T7), U{rf) = \im.^ , W,(r ? ) = lim^ -. 

They are evidently linear operators in rj. 
Lemma 3.1 We have 

[A'(0) V }a s = -[A-f'(a s (r))}[a' s (r)M( V )] in B 3 , (3.5) 

[B'(0)r)} ■ v a = - V ■ [v' s (r)M( V )e r ] + g' {a s )a' s {r)M{r,) in B 3 , (3.6) 

- [A'mvs + [B'(0) V ]p s ~[B\0) V ]g(a s ) = ^V[g'(a s )a' s (r)M( V )} 

-V[ P ' s (r)M( V )}+A[v' s (r)M( V )e r } in B 3 , (3.7) 



[B'(0)rj]®v s + ([B'(0)r]}®v 



n = 



+ 



V ® [v' s (r)M(7])e r ] + ( V ® [<(r)M(r ? )e r ] j j n 
p'.W + ^'^KM-^l^no on S 2 , (3.8) 



[P'(0)r ? ]v = -tr §2 (v) • V w r/ in B 3 , 
[^(r)M(r ? )e r + J7(r/)v s ] cfe = 0, 



X <1 



/ v s -W j (r ] )dx = 0, j = l,2,S. 
J\x\<l 



(3.9) 
(3.10) 

(3.11) 



Proof: (3.5) follows from (5.8) of [2]. To prove (3.6) we denote a St€r] = a s o and 
v s,e?7 = v s ° ^e^- By making Hanzawa transformation to the equation V • v s = g{a s ) we have 

Bierf) ■ v Sj£r , = g(a s>ev ). (3.12) 
Since v S)eJ? | e= o = v s and <7 s , e77 |e=o = c s , we get 

[B(er?) - 23(0)] • v Sy€V + B(0) ■ [v s>€V - v s ] = g(a Sitv ) - g(a s ). 
Dividing both sides with e, then letting e — > and using the relations 

l im ^ ~ ^ = </ a (r)Mfa), lim V ^ ~ Vs = <(r)M(r ? )e r , (3.13) 



£^0 



we see that (3.6) follows. To prove (3.7) we denote p s ,en = Ps $en- By making Hanzawa 
transformation to the equation — Av s + X7p s — ^V(g(a s )) = we get 



A(er])v Sy£V + B(ei])p s ^ v - -B(erj)(g(a SjeT] )) = 0. 



Since p Sy€V \e=o = Ps and 



Ps,erj Ps 



lim 

e^O e 



p' s {r)M{ri), 



(3.14) 
(3.15) 



by a similar argument as before we obtain (3.7). 

Next we prove (3.8). We denote = e r o <3? er? . By making Hanzawa transformation to the 
equation T(v s ,p s )| g2 no = — 7no and noticing that e r | g2 = no, we get 



T« 7 (v a , OJ ,p S) ^)S^ + jef = on $~ i (§ 2 ). 



(3.16) 



By (3.13) and (3.15) we have v S)€r? = v s + eUg(r)M(r/)e r + o(e) andp Sj£r; = p s + ep' s (r)M(rj) + o(e). 
Thus 



® v s , e „ + S(e77) ® v 



\e;/ 



2 - 

Ps,eri 



B(0) ® v s + (b(0) ® v, 



p s + -B(0) • v £ 



+r<j;6 v '(())//].: v. s +([#(0)t/]®v,) T -|[#(0)»;]-v.I 



+B(0) ® [<(r)M(r7)e r ] + \ B{0) ® [^(r)M(r/)e r 
-^(r)M(r/)I - -23(0) • [<(r)M(r,)u,]l} + o(e). 



Noticing that 

[B(0) ® v s + (23(0) ® v s ) T ] - [ Ps + |fi(0) • v s ]I = T(v„p a ), 

and denoting by L(r/) the expression in the braces, we see that the above result can be briefly 
rewritten as follows: 

T eJ7 (v s , e?? ,p s , er? ) = T(v s ,p s ) + eL(r?) + o(e). (3.17) 
Since for x E ^^(S 2 ) we have 

<& erj (x) = x + ex(r — l)Hx(r])(x)x/r = [r + e%(r — l)ili(?7)(x)]u;(x), 

where w(x) = x/r, and 3> €r? (x) 6 § 2 > we see that 3> eJ? (x) = u;(x) for all x € ^^(S 2 ). This implies 
that er V (x) = e r (uj(x)) = no(u(x)) for all x G ^^(S 2 ). Hence, from (3.16) and (3.17) we get 

= [ r T e ri('Vs,€rijPs,eri) e r ri + l e r^\ | 0- 1 (§2) 

= [T(v Sl p s )noow + 7 noow]|^ 1(§2) + e^fa)!*- 1 ^ + °( e ) 



l-f E V(s 2 ) 



9 



T(v s ,p s )n + 7n - e r/— ( T(v s ,p s )n ) ^ o + eL(r?) ^ + o(e). 



(3.18) 



(J = w(x) 



Points on $ (§ ) and S such that the last equality holds are related by the relation 
for x E ^^(S 2 ) and u E S 2 , and in getting the last equality we used the following relations: 

[T(v s ,p s )n o u + 7n o u>] | $ -i ( g 2 ) 



9 / 

T(v s ,p s )n +7n - e rj— \T(v s ,p s )n ) ^ + o(e), 



L (»/)U-x (sa) =I*(»7)|s2 + 0(e). 



The proof of the first relation uses a similar argument as that used in (4.29) of [10], and the 
second relation is immediate. Since [T(v s ,p s )no + 7110] |§2 = 0, M(?y)|g2 = r], and by the result 
in Appendix A of [H] we have 



— (T(v s , Ps )n )| s2 = [2<(r) - p' s (r) - -g'(a s )a' s (r)]\ r= n = -4 5 (l)n , 



by dividing (3.18) with e, then letting e — > and using (3.6), we see that (3.8) follows. 

Finally, (3.9) is immediate, and (3.10), (3.11) follow from the relations / v s dx = 0, 

/ v s x idi = and a similar argument as above, which we omit here. This completes the 

JS 2 



proof of Lemma 3.1. 



□ 



Lemma 3.2 For v E C m+e (B> 3 ) and rj G C m+e (§ 2 ) we /wue 



D a V{a s ,0)v =V{0)[g'((T s )v} + -Q(0){V[ 5 »]}, 



(3.19) 



D p V(a s ,0) V = v' s (r)M(r,)e r - V(0)[g'(a s )a' s (r)M(r,)] - -Q(0){V[g'(a s )a' s (r)M(r,)]} 



+R(0) [7(7/ + -A w 7 ? )n - 2 9 (l)V w r ? + 4 9 (l)7 ? n ] . 



(3.20) 



Proof: From the definition of V(o~, p) it is clear that 

DzV(a s ,0)v = P(P)W(v,)v] + ^Q(0)B{0)[g'(a s )v). 

Since B(0) = V, we see that (3.19) follows. 

To compute V = D p V(a s , 0)rj we denote v = V(cr s ,e7?), where rj G C m+e (S: 2 ) is given. By 
the definition of V(a,p) we see that there exists a function p G C m ~ 1+e (M ) such that (v,p) is 
the unique solution of the problem 

B(erj) ■ v = g{a s ) in B 3 , (3.21) 

-A(er))v + B(eri)p=^B{erj) {g(a s )) in! 3 , (3.22) 
T er) (v,p)n e7? = -7K e7? n £7? on § 2 , (3.23) 
/ v(x)G ev (x)dx = 0, (3.24) 

J\x\<l 

v(x) • wf (x)G er ,(x)dx = 0, j = 1,2,3. (3.25) 

'N<i 

We note that the above problem does have a unique solution. Indeed, from the proof of (2.34) 
of [H] we see that for any sufficiently small e > 0, the conditions (2.24) and (2.25) are satisfied 
by V 9 = g{°~s), g = \B{er])(g{o- s )) and h = — jK eri n eri with p = erj. Hence the desired assertion 
follows from Lemma 2.2. 

Clearly, lim e ^o v = v s , lim e _ > oP' = Ps aim V = lim^o e X (v — v s ). Hence, by a similar 
argument as in the proof of (3.6) and (3.7) we get, from (3.21) and (3.22) respectively, that 

V • V = -[B'(0)r]} ■ v s in B 3 , (3.26) 

and 

- AV + VP = [A'(0)v]v s - [B'(0)v]p s + -[B'(0) V ]g(a s ) in B 3 , (3.27) 

where P = lim e ^o e l {p ~ Ps)- Next, recalling that 

n p (x) = n((f> p (x)), k p {x) = k(4> p (x)) for x G § 2 , 

where as before n and k are the unit outward normal and the mean curvature of T p (= p (S 2 )), 
respectively, by a direct computation we easily obtain 

ii ev = n - eV w r] + o(e) and K erj = 1 - e[r? + 7^77] + o(e). 

Thus similarly as in the proof of (3.8) we get from (3.23) that 

[V®V + (V® V) T ] n =-{ [B' (0)??] ® v s + ([#(0)77] 8> v,) T }n 

+7{V^r ? + [rj + -A^no} + Pn + T(v s ,p s )V w r/ on § 2 . 



A direct computation shows that (cf. (4.33) of [TO 



T(v s ,p s ) V^r? = -(7 + 2 5 (l))V a ,r/. 
Hence by using (1.11), (3.26) and the above result we obtain 

T(V, P)n = [V ® V + (V <g> V) T ] n - [P + • V]n 

= -{[B'(0) V ] ®v s + ([^(0)77] ® v s ) T }n + |{[#(0)tj] • v s }n 

+ 7 [t? + -A^no - 2g(l) on S 2 . (3.28) 
Finally, similarly as in the proof of (3.9) and (3.10) we get from (3.24) and (3.25) that 

/ [D p V(a s ,0)rj + U(rj)v a ] dx = 0, (3.29) 

J\x\<l 

[ {[D p V(a s ,0)r l ]-w j + v s -W j (ri)}dx = 0, j = 1,2,3, (3.30) 

J\x\<l 

respectively. Now let Vi = V- v' s (r)M(r])n and Pi = P - p' s (r)M(rj). Then from (3.6)-(3.11) 
and (3.26)-(3.30) we easily obtain 

V • V! = -g'(a s )a' s (r)M( V ) in B 3 , (3.31) 

- AVi + VPi = ~V[g'(a 8 )a' s {r)M{r,)] in B 3 , (3.32) 

T(V 1 ,P 1 )n = 7(??+^A^)no-2 5 (l)V w ? ? + 4 9 (l)7 ? no on § 2 , (3.33) 

Vi dx = 0, (3.34) 

x\<l 



I V 1 -w j dx = 0, j = 1,2,3. (3.35) 

J\x\<l 

In getting (3.35) we also used the fact that no • Wj = for j = 1,2,3. Using the relations 
/ V^ry • Wj dS u = — rjS/ w • Wj dS^ = (j = 1, 2, 3) we easily see that the relations (2.24) 

J\x\=l ' J\x\=l 

and (2.25) with p = are satisfied by 

if = -</(a s K(r)M(77), g = ~V[g'(a s )a' s (r)M( V )}, 

and 

h = 70? + 2 A^) n o - 25(l)Vo;?7 + 45(l)r?n . 

Hence by Lemma 2.2 we see that the problem (3.31)-(3.35) has a unique solution (V\,Pi) and, 
in particular, Vi is given by 

V a = -P(0)[g>(a s y s (r)M( v )] - ±Q(0){ V[g'(a s )a' s (r)M( V )}} 

+R(0) [ 7 (ry + l-A^no - 2g(l)V wV + 4 5 (l)r?n ] , (3.36) 



from which (3.20) immediately follows. This completes the proof of Lemma 3.2. □ 

We are now ready to compute all the Frechet derivatives appearing in the right-hand side 
of (3.1). First, by (2.31), (3.3) and (3.19) we have 

D a g(a s ,0)v = -5(p)[D s V(*„0)v] = tv S 2{P(0)[g'(a s )v] + ^Q(0)[V( 5 ' (a s )v)}} ■ n . (3.37) 

Next, by (3.9) and the facts that V(a s , 0) = v s , tr g2 (v s ) = we have 

[D'(0)r}]V{a s ,0) = -tr §2 [V(a s , 0)] • V w r/ = -tr§ 2 (v s ) • V w r/ = 0. 

Thus by (2.31), (3.3) and (3.20) we have 

D p G(a s , 0)7j = [V'{0)rj]V(a s , 0) + V(0)[D p V(a s , 0)r?] 
= tT § 2[D p V((J s ,0)r]} ■ n 

= ^(1)7? - tr §2 {^(0) [g\a s y s (r)M(r))] + ^Q(0){V[g'(a s )a' s (r)M(r,)}} 

-R(0) [7(7/ + ^A^no - 2g(l)V ojV + 4 5 (l)r/n ] } • n . (3.38) 
Thirdly, from (2.30) and a direct computation we have 

D*F(a s , 0)v = e"U(0)« - e- l f{a s )v + x (r - l)n 1 {P(0)[ J D ? V(a s , 0)t;]}/3(0)<7 s • e r 

+ X (r - l)ni (V(0)V(a s , 0))B(0)v ■ e r 
= e-^A - f'(a s )]v + X (r - ^{D^D^Via,, 0)v]}a' s (r) 

+X{r - l)ili[tr §2 (v s ) • n ]Vt; • e r 
= e'^A - f'(a s )]v + X (r - l)^(r)n 1 {tr s2 {P(0)b , (a s ) W ] 

+±Q(0)[V(g'(a s )v)}}-n Q }. (3.39) 

Finally, from (2.30), (3.5), (3.38) and a direct computation we have 

D p F{cj s , 0)77 = e-^A'iOWa, + x(r - l)^^^, 0) V ]B(0)a s ■ e r 
+ X (r - l)n 1 [P(0)V(a s ,0)][/3 / (0)r/]a s • e r 
= e- l [A\V)r{\o s + X (r ~ mi[D p g{a s ,Q>)r{\a' s {r) 
(because V(0)V(a s ,0) = tr§ 2 (v s ) • no = 0) 
=_ e -i[A - f'(a s )][x(r - 1)^(0^(7?)] + g(iy s (r)M(n) 
-X^-lKWnijtrg. [v(0)[g'(a s )a' s (r)M( V )] 

+lQ(0){V[g'(a s )a' s (r)M(7 ] )]} - R(0){ 7 (r/ + ^A^)n 

-2< 7 (l)V w r / + 4<7(l)r / n }] -n }. (3.40) 

In conclusion, we have the following lemma. 

Lemma 3.3 The Frechet derivative D¥(U S ) is given by (3.1), in which D^F{a s ^), 
D p J r (a s ,0), D^Q(a s ,0) and D p Q(a s ,0) are given by (3.39), (3.40), (3.37) and (3.38), respec- 
tively. □ 



As usual, for a linear operator L from a product space X% x X2 to another product space 
Y\ x Y 2 having the expression 



we write it as L = ( 11 12 ] . Then we have 



L(xi,x 2 ) = (Luxi + Li 2 X2,L 2 iXi + L22X2) for (xi, x 2 ) G Xi x X 2 , 
Ln L\ 2 

L21 L22 / 

D¥(U S ) 



D^g(a s ,Q) D p g(a s ,0) 



In the sequel we follow the idea of [2] to study the property of this operator and compute its 
spectrum. 

Recall that m > 3, m G N and d G (0,1). Let A : C m+9 (B 3 ) -» C m - 2+e (B 3 ) and 
J : C m+e (B 3 ) -> C m+e (§ 2 ) be the following operators: 

•4 v = [A - /'(a-)]" for u G C m+9 (B 3 ), (3.41) 

Jv = tr s2 + ^Q(0)(V(<7»))] • n for « G C m+9 (M 3 ). (3.42) 

Clearly, A G L(C m+0 (B 3 ), C m - 2+e (B 3 )), and 

J G L(C m+e (B 3 ), C m+e (S 2 )) n L(C m ~ 2+e (B 3 ), C m - 2+e (S 2 )). (3.43) 

Next let n : C m ~ 2+e (S 2 ) -► <7 m - 2+e (B 3 ) be the operator defined by n (r?) = u for 77 G 
C7 m - 2+e (S 2 ), where u G C m ' 2+e (B ) is the unique solution of the following boundary value 
problem: 

[A - f'(a s )}u = inB 3 , u = 77 on S 2 . 

It is clear that n G L(C m ~ 2+d {S 2 ), C m ~ 2+e (M 3 )) n L(C m+e {$ 2 ), C m+e (B 3 )) and A U = 0. We 
also let /3 7 be the following operator from C m+e (S 2 ) to C m ~ 1+e (S 2 ): 

B lV = 5(1)77 - a' s (l)JU ( V ) + tr g2 |r(0) [7(77 + ^77)110 

-2 5 (l)V w r ? + 4 5 (l)7?n ] } • n for 77 G C m+e (S 2 ). (3.44) 

It is evident that B 7 G L(C m+9 (§ 2 ), C m - 1+e (S 2 )). Clearly, 

f D^fo, 0)u = e" 1 A« + x(r - l)a' s (r)UxJv, 
D^g(a s ,0)v = Jv, 

D p T(a s , 0)t? = X (r - lK(r)IIifa) [B jV + a' s (l)JU ( V ) - J(x(r - 1)^^)^(17))] (3.45) 
-e- 1 A \x(r-l)a' a (r)Ui(T l )], 
{ D p g(a s ,0) V = B jV + a' s (l)jn ( V ) - J(a' s (r) X (r - l)n x (»/)). 

Finally, let M : Xo — > X and T : X — > X be the following operators: 

/ rU + a' s (l)U J a' s (l)U B 7 \ fl -^(l)n + x(r - l)^(r)nA 



It is easy to see that M G L(Xo,X) and T G L(X). Moreover, since tr S 2{ — a' s (l)Uo(r]) + x{ r — 
lK(r)IIi(?7)} = -a' s (l)rj + a' s (l)rj = for any 77 G C m+e (8 2 ), we see that T maps X into itself, 
i.e. T G L(X) n L(Xq). Besides, it can be easily seen that 



By a simple computation we have 

Lemma 3.4 D¥(U S ) = TMT -1 . □ 

Given a closed linear operator L on a Banach space, we denote by o~(L) and o~ p (L) respec- 
tively the spectrum and the set of all eigenvalues of L. As an immediate consequence of Lemma 
3.4 we have the following preliminary result: 

Lemma 3.5 (i) Regarded as an unbounded linear operator in X with domain Xo, D¥(U S ) 
is an infinitesimal generator of an analytic semigroup in X. 

(ii) If 5 is sufficiently small then for any U G 0$ and in a neighborhood ofU s in Xo, we have 
that D¥(U), regarded as an unbounded linear operator in X with domain Xo, is an infinitesimal 
generator of an analytic semigroup in X. 

(Hi) Let V G X and A G C. Then D¥(U S )V = XV if and only i/M(TrV) = AT _1 V. 

(iv) a(D¥(U s )) = a p (D¥(U s )) = a p (M) = <r(M). 

Proof: (i) By Lemma 3.4 it suffices to prove that the operator M, regarded as an unbounded 
linear operator in X with domain Xo, is an infinitesimal generator of an analytic semigroup in 
X. We denote 



Then M = Mi + M 2 . From (3.43) it can be easily seen that M 2 G L(X). Thus by a standard 
result for perturbations of generators of analytic semigroups (see [E]), we only need to show 
that Mi, regarded as an unbounded linear operator in X with domain Xo, is an infinitesimal 
generator of an analytic semigroup in X. 

Clearly, the operator e~ 1 Ao + o~' s (l)TloJ is an infinitesimal generator of an analytic semi- 
group in C m - 2+e (E 3 ) (with domain C m+9 (JB 3 )). Next, from (3.42), (3.44) and the definition of 
lEo it can be easily seen that the operator can be rewritten in the following form: 





B^r] = tr&(v) ■ n + g(l)rj for r, G C m+e (S 2 



where v is the second component of the solution (</>, v, of the following problem: 



= f'(a,)<f> 



m 



3 



V -v = g'{o- s )(t) 



m 



3 
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B' 



3 



-a' s {l)r] on § 2 , 



T(v, ^)n = -2 5 (1)V W ?? + 7(77 + -A w r/)n + 4 5 (l)77n on S 2 , 



v dx = 0, 



N<i 

tJxi(ii = 0. 



S <1 



This shows that /3 7 is the same operator as that given by (3.9) of [13] with the same notation. 
Thus by Lemma 2.6 and Lemma 3.1 of [2] we see that is an infinitesimal generator of an 
analytic semigroup in C m ~ 1+e (S 2 ) (with domain C m+e (§ 2 )). Besides, for any rj 6 C m+e (§ 2 ) we 
have 

lks( 1 ) n o^7 ? ?llc™-2+9(I 3 ) - CH^y^llc"™-^^ 2 ) < C||^7^|lc ,m - 1 + s (S 2 )! 
i.e., O"g(l)llo0 7 is i3 7 -bounded in the notion of [13]. Hence, by Corollary 3.3 of [13] (see also 
Lemma 3.2 of pQ) we see that Mi is an infinitesimal generator of an analytic semigroup in X 
(with domain Xo), as desired. This proves the assertion (i). 

The assertion (ii) is an easy consequence of the assertion (i) and the fact that F £ 
C 0O (O,5,X), and the assertion (iii) is immediate. Finally, since Xo is compactly embedding 
into X, the assertion (iv) follows from (i) and (in). The proof is complete. □ 

Later on we always assume that 5 is sufficiently small such that the open set O5 satisfies 
the condition of Lemma 3.4 (ii). 

As in [14], for every integer I > we let Yi m (cu), m = —I, — 1 + 1, • • • , I — 1, 1, be a normalized 
orthogonal basis (in L 2 (S 2 ) sense) of the space of all spherical harmonics of degree I. It is 
well-known that 

\>Y lm (w) = -{I 2 + l)Y lrn {uo). 
For every such I we denote by F[(r) the unique solution of the following ODE problem: 



if (r) + far) - l ^ L F l (r) = /'(a s (r))F ; (r) for < r < 1, 
F/(0) = 0, F,(l) = -^(1). 



Next we denote 



«/ (7) 



«o = 5(1) + / g'{a s {r))F (r)r 2 dr, 
Jo 

l(l + 2)(2l + l) 



4(2Z 2 + 4/ + 3) 



(7-7l)i for I > 2, 



where 
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4(2Z + 3)(Z + 1) 



l{l + 2)(2l + 1 
From Lemma 3.2 and Lemma 3.3 of |14! we have: 



5 (1) + / </(a s (r))i^(ry+ 2 dr 



Z > 2. 



(3.46) 

(3.47) 
(3.48) 

(3.49) 



Lemma 3.6 (i) /3 7 is a Fourier multiplication operator having the following expression: 
For any 77 E C°°(§ 2 ) wii/i Fourier expansion t](lo) = YluLo EL=-l ci m Yi m (uj), we have 

00 / 

B 7 n(u;) = a cooY 00 + ^ ^ ai(^)ci m Yi m (uj). 



(ii) The spectrum of is given by 

f 7(S 7 ) = {a ,0}U{a / ( 7 ):Z = 2 ) 3,4 ) ---}. 
Moreover, the multiplicity of the eigenvalue is 3. □ 

By Lemma 3.4 (ii) of [2] we know that 7; > for all I > 2, and lim^oo 7/ = 0. Thus as in 
[2] we define 

7* = max 7;. 

l>2 

Clearly < 7* < 00, and for 7 > 7* we have ety( 7 ) < for all I > 2, while if 7 < 7* then 
there exists I > 2 such that 0/(7) > 0. Since clearly lim^^QQ 

a l(l) = tne following notation 

makes sense: 

a* = max{ao,ai( r y),l > 2}. 

By Lemma 3.4 (i) of [2] we know that ao < 0. Thus a* < for all 7 > 7*. 

In the following lemma e is the constant appearing in the equation (1.1), which also appears 
in the expressions of D¥(U S ) and M. 

Lemma 3.7 We have the following assertions: 

(i) is an eigenvalue o/M of multiplicity 3. 

(ii) For any 7 > there exists a corresponding constant e' > and a bounded continuous 
function /// 7 defined on (0, e' ], such that for any < e < e' we have 

a(M)D{A l)7 (e) = 0,(7) +^1,7®, 2 = 2,3,---}, (3.50) 
and /or eac/i / > 2, the eigenvectors of M corresponding to the eigenvalue Aj i7 (e) have the 

(SCL[ (r, £J \ 
' '"l ) ^ m ^' w ^ ere a i,7( r ' e ) * s a smooth function in r G [ro, 1] and is 

bounded continuous in e G (0, e ] . 

(iii) For any 7 > 7* i/iere exists a corresponding constant eq > sitc/t i/iaf /or any < e < 
£0, we /iaue 

sup{ReA : A G cr(M)\{0}} < ^a* < 0. (3.51) 

Proof: We assert that for a vector U = (v,rj) £ Xo, MC7 = if and only if t> = and 
£> 7 77 = 0. Indeed, it is easy to see that M.U = if and only if Aov = and £> 7 r/ = 0. Since 
U G X implies that v G C m+e (5 ) n Co (B ) , we see that the boundary value of v is zero. Hence, 
by the maximum principle we see that .4.0^ = implies that v = 0. This proves the desired 
assertion. By this assertion and the fact that is an eigenvalue of i3 7 of multiplicity 3, we 
immediately get the assertion (i). Next, by making slight changes of the proof of Lemma 6.4 
of [2], we get the assertion (ii). Finally, the assertion (iii) follows from a quite similar proof as 
that of Lemma 6.5 of [2]. □ 



4 The proof of Theorem 1.1 



In this section we give the proof of Theorem 1.1. By Lemmas 2.1 and 2.3, we only need to 
prove that the stationary point U s of the equation (2.34) is asymptotically stable in case 7 > 7* 
whereas unstable in case 7 < 7*. By Lemma 3.5 (ii) we see that the equation (2.34) is of the 
parabolic type in a small neighborhood of U s . It is thus natural to use the geometric theory 
for parabolic differential equations in Banach spaces to investigate this equation. In doing so, 
however, we meet with a serious difficulty that, by Lemma 3.5 (iv) and Lemma 3.7 (i), is an 
eigenvalue of D¥(U S ), so that the standard linearized asymptotic stability principle cannot be 
used to tackle the equation (2.34). We shall as in [2] appeal to the Lie group action possessed 
by this equation to overcome this difficulty. 

For t > we denote by B 3 the ball in M 3 centered at the origin with radius r. Regarding B 3 
as a neighborhood of the unit element of the commutative Lie group M 3 , we see that G = B 3 is 
a local Lie group of dimension 3. Given z E M 3 , we denote by S z the translation in M 3 induced 
by z, i.e., 

S z (x) = x + z for x E M 3 . 

Let p E C 1 (§ 2 ) such that H/jIIc* 1 ^ 2 ) is sufficiently small, say, ||p||ci(s 2 ) < ^ f° r some small 5 > 0. 
For any z E B 3 , consider the image of the hypersurface r = 1 + p(u) under the translation S z , 
which is still a hypersurface. This hypersurface has the equation r = 1 + p(u>) with p E C 1 (S 2 ), 
and p is uniquely determined by p and z. We denote 

p = S*(p). 

By some similar arguments as in the proof of Lemmas 4.1 and 4.3 of [2] we can show that if p E 
0™ +e (S 2 ) then S*(p) E C m+0 (S 2 ) and S* E C(0™ +e (S 2 ), C m+e (S 2 ))nC 1 (0™ +e (S 2 ), C m - 1+9 (S 2 )) 

Next, for each z E B 3 and p E C 1 (S 2 ) such that 1 1 /o| | c 1 (s 2 ) i s sufficiently small, let P ZtP : C(B ) — » 
3 

C(B ) be the mapping 

P,»(s) = u^^p)^) - *)) for « E C(B 3 ). 

3 

This mapping is well-defined and, actually, we have P Z:P E L(C(B )). Indeed, letting p = S*(p) 
and denoting by £l p and £l p the domains enclosed by the hypersurfaces r = 1 + and 
r = 1 + /5(o;), respectively, we see that x E B 3 if and only if <& s *( p ^(x) = $ p (x) E Cl p , which 
is equivalent to ^> s *^(x) — z E Q p . But we know that y E fi p if and only if $~ 1 (y) E B 3 . 
Hence, we see that x E B 3 if and only if $~ 1 ( < I>5*(p)(x) — z) E B 3 . Moreover, it can be easily 
seen that for p E C 1 (§ 2 ) the mapping x — ► $~ 1 ($5»(p)(2;) — z) is a C 1 -diffeomorphism from B 
onto itself. Hence the desired assertion follows. Furthermore, it is also immediate to see that 
if p E 0^ +6> (S 2 ) then the mapping x — ► $~ 1 (<I>5»(p)(x) — z) is a C m+e -diffeomorphism from B 3 
onto itself, so that P z>p E L(C m+0 (B 3 )). 

Now, for each z E G = B 3 we define a mapping S* : O s -> X (recall that 5 = (C m+9 (B 3 )n 
C (B 3 ))xO^ +e (S 2 ) andX = (C m+e (B 3 )nC (B 3 )) x C m+0 (S 2 )) as follows: For any U = {u,p) E 
O s , let 

S* z (u,p) = (P z , p (u),S* z (p)). 



It is obvious that if u E Co(B )) then also P ZtP (u) E Cq(M )). Thus for r and 5 sufficiently small 
this mapping is well-defined and it does map Og into Xq. We have 

Lemma 4.1 Let m > 3 and < 6> < 1. 7e£ 

= (c m ~ 2+d 3 ) n C (B 3 )) x o 5 m - 1+e (s 2 ) C X. 

For sufficiently small r > and 5 > we have the following assertions: 

(i) For any z E M 3 T we have S* E C(Q' S ,X) n C(O 5 ,X ). 

(ii) For any z, to E B^ we have 

S* o S* = S* z+W , S* = id, and (S*)" 1 = S^. 
(Hi) The mapping S* : z — > S* /rora B^ to C(0^,X) is an injection, and 
S* E C fc (B^, C'(0 5 , C m - fc -' +0 (B 3 ) x C m - fc - i+9 (S 2 ))), fc > 0, I > 0, k + I < m. (4.1) 
Moreover, for fixed z E B^ we aaue 

E C(Q 5 , L(C m - 1+e (B 3 ) x C m - 1+e (S 2 ))), (4.2) 

i.e., /or any £/ E 0,5, i/ie operator DS*(U) (which is, by (4.1), an bounded linear operator from 
C m+e (B 3 ) x C m+e (S 2 ) to C m ~ 1+e (n 3 ) x C m - 1+e (S 2 )) can be extended into a bounded linear 
operator from C m - 1+e (M 3 ) x C m - 1+e (S 2 ) to itself, and the mapping U -> £>S*(f7) /rom O s to 
L(C m ~ l+e (M 3 ) x C m - 1+9 (S 2 )) is continuous. 

(iv) Define p : M 3 x Q' s -» X by p(z,U) = S*(J7) /or (z, 17) E B^ x 0' 5 . Then for any 
U E Q«5 we have p(-, U) E C^B^X), and r&nkD z p(z, U) = 3 /or every z E B^ and U E 5 . If 
furthermore U E Z = C°°(B 3 ) x C°°(S 2 ) £aen 17) E C°°(B3,Z). 

Proof This lemma follows from a similar argument as that in the establishment of Lemma 
4.4 of [2]. Indeed, to get the assertions of this lemma we only need to replace the Sobolev and 
corresponding Besov spaces W m ' q (M 3 ), B m - 1 - 1 /i(S 2 ) in [2\ with the Holder spaces C m+9 (M 3 ) 
and C m+9 (S 2 ), and after a such replacement all the analysis presented in Section 4 of [2] still 
holds. In particular, the proof of (4.1) uses a similar argument as that of the second assertion in 
(i) of Lemma 4.3 of [2]. Since this analysis is lengthy but does not have new ingredient different 
from that in [2], we omit it here. □ 

Corollary 4.2 Let assumptions be as in Lemma 4.1. Then for sufficiently small r > 0, 
5 > and fixed z E B^ we also have the following relation: 

DS* Z E C(O s ,L(C m - 2+e (W 3 ) x C rn - 1+e (S 2 ))). (4.3) 

Proof. Since S*(u,p) = (P Z)P (u), S*(p)), we see that 

DS* z (u,p)(v, V ) = (D u P ZiP (u)v + D p P z>p (u) V ,DS* z (p) V ). 

From (4.2) we have \(u,p) -> DS*(p)] E C(O s , L(C m ~ 1+e (S 2 ))) and [(u,p) -> D p P ZjP (u)] E 
C(O s ,L(C m - 1+e (S 2 ))) C C(0 5 ,7(C m - 1+e (§ 2 ),C m - 2+e (S 2 ))). Since m-1 > m-2, by (4.2) we 



also have [(u, p) — > Z? u P Zi p(n)] G C(©5,L(C m 2+e (B 3 )). Combining these assertions together, 
we see that (4.3) follows. □ 

In the sequel, for p = p(t), u = u(x,t) and U = (u(x,t), p{t)), we denote by P ZjP (u) the 
function u(x,t) = u(<&~^(<& s*(p(t))( x ) ~ by S*(p) the function p(t) = S*(p(t)), and by 

S* Z (U) the vector function (P 2>p (u), S*(p)) = (u(x , t) , p(t)) . 

Lemma 4.3 IfU = (u, p) is a solution of the equation dU/dt = ¥(U) such that WpWc 1 ^ 2 ) is 
sufficiently small, then for any z£R 3 such that \z\ is sufficiently small, S* Z (U) = (P ZtP (u), S*(p)) 
is also a solution this equation. 

Proof. It is easy to see that if (a, v,p, Q) is a solution of the system of equations (1.1)— (1.8), 
then for any z € M 3 , we have that (<r, v,p, Cl) defined by 

a(x,t) = a(x — z,t), p(x,t) = p(x — z,t), v(x, t) = v(x — z, t), O(i) = J7(t) + z, 

is also a solution of (1.1)— (1.8). From this fact we see immediately that if U = (u, p) is a solution 
of the equation 

§=nu } , («) 

then U = (u,p), where 

u(x,t) = u(^ t) ^ s:im (x) -z),t), p(t) = S* z (p(t)), 
is also a solution of this equation, which is the desired assertion. □ 

Lemma 4.4 If r and 5 are sufficiently small then for any z £ B 3 and U = (u, p) £ 0$ we 

have 

¥(S* Z (U)) = DS* Z (U)¥(U). (4.5) 

Proof. By Lemma 3.5 (ii) we see that the equation (4.1) is of the parabolic type in Os, 
provided d is sufficiently small. Hence, by a well-known result in the theory of differential 
equations of the parabolic type in Banach spaces (cf. Theorem 8.1.1 of [12]) we see that given any 
Uq = (u, p) G Xo there exists to > such that the equation (4.1) has a unique solution U = U(t) 
for < t < t , which belongs to C([0,i ],X) nC , ((0,*o] J 0^)nL 00 ((0,t ),Xo)nC ,1 ((0,to],X) and 
satisfies the initial condition U(0) = Uq. Let U(t) = S* z (U(t)) for < t < t . By Lemma 4.2, U 
is also a solution of (4.1), satisfying the initial condition U(0) = S*(Uo). The fact that U is the 
solution of (4.1) implies that 

^1 = ¥(U(t)) for < t < t . 
dt 

On the other hand, since U(t) = S z (U(t)), we have 

^ = DS* z (U(t))^ = DS* z (U(t))¥(U(t)) for < t < t . 



Thus 



¥(U(t)) = DS* z (U(t))¥(U{t)) for < t < t . 



(4.6) 



KU(t) is a strict solution (in the sense of [H]) then U G C([0,f ],O j )nC 1 ([0,t ],X) and clearly 
U(t) is also a strict solution, so that by directly letting t — > + we get 

F(S:(C/ )) = DS* Z (U )¥(U ). (4.7) 

If C/(t) is not a strict solution then we establish this relation in the following way: Let 

Xo = C m - 2+e (B 3 ) x C m ~ 2+e (§ 2 ), X = C m - 4+9 (B 3 ) x C m - 3+e (S 2 ), 

and let ©5 be a small neighborhood of the origin of Xo. Then F G (7(0,5, X) an d DS Z € 
C(6 5 ,L(X)) (by (4.3)). Hence, since U G C([0,t Q ],X) C C([0,t ],X ), which also ensures that 
£/ G C([0, i ], X ), by letting i -> 0+ we see that (4.7) still holds. □ 

We are now ready to prove Theorem 1.1. 

Proof of Theorem 1.1: We first assume that 7 > 7*. Then by Lemma 3.5 (iv) and 
Lemma 3.7 (i) and (Hi) we see that 

sup{ReA : A G a(D¥{U s ))\{0}} < -a* < 0, 

and is an eigenvalue of D¥(U S ) of multiplicity 3. By Lemma 4.1 we see that the mapping 
p:B^xO^X defined by p(z, U) = S* Z {U) for (z, U) G B 3 x ©^ is a Lie group action of the 
local Lie group G = B 3 to the space X, and by Lemma 4.4 we see that the equation dll/dt = ¥(U) 
is quasi-invariant (in the sense of [2]) under this Lie group action. Since for every z G B 3 and 
U G O5 we have iwkD z p(z, U) = 3 = the multiplicity of the eigenvalue of D¥(U S ), we see 
that all the conditions of Theorem 2.1 of [2] are satisfied by the equation dll/dt = ¥(U) and 
the Lie group action (G,p). Hence, by Theorem 2.1 of [2] we conclude that there exist two 
neighborhoods Oi and O2 of U s in Xo, Oi is larger than ©2 (i.e. ©2 Q ©1), such that for each 
Uo G ©i the initial value problem (2.34) has a unique solution U = U(t) for all t > 0, which 
belongs to C([0, 00), X) n C((0, 00), ©i) n L°°((0, 00), X ) n C7 1 ((0 J 00), X), and for any U G ©2 
there exists corresponding Vq G ©i and z G G = B 3 which are uniquely determined by Uo, such 
that 

U = S* z {Vo), lim Ufa V Q ) = U s , and lim U(t; U ) = S* Z (U S ), 

t— >oo t— >oo 

where U(t;Vo) refers to the solution of (2.34) with initial value Vb, and similarly for U(t;Uo). 
Moreover, the convergence in the above limit relations is exponentially fast. Prom this result 
and Lemmas 2.1 and 2.3 we immediately obtain (1.13). 

Next we assume that 7 < 7*. Then there exists I > 2 such that a; (7) > 0. By Lemma 
3.5 (iv) and Lemma 3.7 (ii) we see that for e > sufficiently small, D¥(U S ) has an eigenvalue 
Xl,y(s) = 07(7) + £fj,i^(e) which is clearly positive. Hence, by a standard result in the geomet- 
ric theory of parabolic differential equations in Banach spaces (cf. Theorem 9.1.3 of [12]) we 
conclude that U s is unstable as a stationary point of the equation dU/dt = ¥(U). Using again 
Lemmas 2.1 and 2.3, we get the last assertion of Theorem 1.1. This completes the proof. □ 
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